An Interactive Virtual Reality Simulation System for Robot Control and Operator Training

Robotic systems are often very complex and difficult to operate, especially as multiple robots are integrated to accomplish difficult tasks. In addition, training the operators of these complex robotic systems is time-consuming and costly. In this paper, a virtual reality based robotic control system is presented. The virtual reality system provides a means by which operators can operate, and be trained to operate, complex robotic systems in an intuitive, cost-effective way. Operator interaction with the robotic system is at a high, task-oriented, level. Continuous state monitoring prevents illegal robot actions and provides interactive feedback to the operator and real-time training for novice users. Contributions of the Paper: This paper presents a summary of the Virtual Reality based, task-level robot control system under development at Sandia National Laboratories in the Intelligent Systems and Robotics Center. The Virtual Reality approach provides a very intuitive interface which enables operators to gain familiarity with complex robotic systems very quickly. This system is unique in that it combines task-level, voice activated robot control with a Virtual Reality based interface. This type of system has applications for training of operators on complex robotic system and robot control during hazardous/remote robot applications.